fix(tokens): raise the two Anthropic ceilings the gateway now honors#110
Merged
Conversation
claude-haiku-4.5 16384 -> 64000 claude-sonnet-4.6 64000 -> 128000 Both were held low on purpose. The gateway clamped haiku to 8192 and sonnet-4.6 to 64000, and asking past a clamp is not free: quotedOutputTokens scales with the ceiling, so a bigger ask inflated the upfront price quote while the reply stayed capped. blockrun#266 corrected the catalog and #268 added the context-headroom guard that raising the ceilings requires; both are deployed and verified live today. Deliberately NOT raised, though the catalog now reads higher: openai/gpt-5.5 32768 (catalog says 128000) openai/gpt-5.4 32768 (catalog says 128000) openai/gpt-5-mini 16384 (catalog says 65536) The catalog is the only evidence for those three, and it was 8x wrong about haiku-4.5. Raising on that basis alone repeats the mistake this table exists to guard against. A test pins them so the next person has to argue with a failing assertion rather than quietly sync them. One existing test had to change. It asserted haiku stays 16384 to prove the static table beats the catalog — valid when the catalog said 8192, but once both sides read 64000 it proved nothing while still passing. Repointed at gpt-5.5, where the disagreement is real and deliberate. Scope note: the raise only takes effect on the escalation path. A normal turn sends min(CAPPED_MAX_TOKENS 16384, ceiling), so 64000 is reachable only after a max_tokens stop escalates. Not exercised end-to-end — that needs a real long generation. The evidence is Anthropic's published limit plus the deployed gateway clamping at 64000, confirmed against the live catalog. Local suite 636 pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #108/#109, unblocked by BlockRunAI/blockrun#266 + #268 shipping to production today.
What changed
claude-haiku-4.5claude-sonnet-4.6Both were held low deliberately. The gateway clamped haiku to 8,192 and sonnet-4.6 to 64,000, and asking past a clamp is not free —
quotedOutputTokensscales with the ceiling, so a bigger ask inflated the upfront price quote while the reply stayed capped. Raising these before the gateway was fixed would have cost money and bought nothing.blockrun#266 corrected the catalog, #268 added the context-headroom guard that raising the ceilings requires (without it, sonnet-4.6's 128,000 inside a 200,000 window leaves 72K for input, and the streaming path has no truncation net). Both are deployed and verified against the live catalog.
Deliberately NOT raised
openai/gpt-5.5openai/gpt-5.4openai/gpt-5-miniThe catalog is the only evidence for these three, and it was 8× wrong about haiku-4.5. Syncing to it on that basis alone repeats the exact mistake this table exists to guard against. A test pins all three, so the next person has to argue with a failing assertion instead of quietly syncing them.
One existing test had to change
gateway catalog never overrides a static max-output entryasserted haiku stays 16,384 to prove the static table beats the catalog. That was valid when the catalog said 8,192 — but once both sides read 64,000, the assertion proved nothing while still passing green. Repointed atgpt-5.5, where the disagreement is real and deliberate, so it tests precedence again.Scope note (what this does NOT do)
The raise only takes effect on the escalation path. A normal turn sends
min(CAPPED_MAX_TOKENS 16384, ceiling), so 64,000 is reachable only after amax_tokensstop escalates. I did not exercise that end-to-end — it needs a real long generation. A short smoke call against haiku succeeds, but that proves nothing about the ceiling: it never sends more than 16,384. The evidence here is Anthropic's published limit plus the deployed gateway clamping at 64,000, confirmed against the live catalog.Also worth knowing
Re-measuring all picker models against the freshly deployed gateway showed the #108 gap-filler working as designed: 7 mismatches on a cold cache, 2 once warm. The 5 that resolved (
glm-5.2,gpt-5.3-codex,glm-5,o3,gemini-3.1-pro) have no static entry and pick up live values automatically. Only entries that exist-but-are-wrong need hand-editing — which is exactly these two.Local suite 636 pass, 0 fail.